Introduction to Machine Learning

Chapter 23: Recommender Systems and Course Recap

Introduction

In this final lecture, we explore Recommender Systems — one of the most impactful applications of machine learning in everyday life. From Netflix movie suggestions to Amazon product recommendations, these systems drive engagement and revenue across digital platforms. We examine two main approaches: content-based filtering and collaborative filtering (user-based and item-based). We then conclude with a comprehensive recap of the entire course, mapping out the machine learning landscape we've traversed together.

Learning Objectives:

2. Theory

2.1 The Long Tail Phenomenon

Physical stores face scarcity of resources — limited shelf space means they can only stock popular items. Online platforms, however, can make everything available. This distinction is known as the Long Tail.

Items (ranked by popularity) Popularity Head (Popular items) Long Tail (Niche items) Physical stores can only offer this Online stores can offer everything

Figure: The Long Tail distribution. Physical retailers focus on the "head" (popular items), while online platforms can serve the entire tail (niche items).

2.2 Types of Recommender Systems

Recommender systems differ mainly in what information they use to decide that two things are similar: the properties of the items themselves, or the behaviour of other users.

Type Approach Example
Content-Based Analyze item properties to recommend similar items Recommend action movies to users who watched action movies
Collaborative Filtering (User-Based) Find similar users and recommend what they liked Users who bought X also bought Y
Collaborative Filtering (Item-Based) Find similar items to those the user liked and recommend them Because you watched Movie A, you might like Movie B

2.3 User-Based Collaborative Filtering

In user-based CF, we define similarity between two users as the correlation between their ratings on items they've both rated. If one user rates a movie higher than their own average, a similar user is likely to do the same.

Pearson Correlation Similarity between users u and v:
$$sim(u,v) = \frac{\sum_{i \in I_{uv}}(r_{ui} - \bar{r}_u)(r_{vi} - \bar{r}_v)}{\sqrt{\sum_{i \in I_{uv}}(r_{ui} - \bar{r}_u)^2} \sqrt{\sum_{i \in I_{uv}}(r_{vi} - \bar{r}_v)^2}}$$
Predicted rating for user u on item i:
$$\hat{r}_{ui} = \frac{\sum_{v \in N_u} sim(u,v) \cdot r_{vi}}{\sum_{v \in N_u} |sim(u,v)|}$$

2.4 Item-Based Collaborative Filtering

In item-based CF, we first determine how similar items are based on user ratings, then predict ratings by giving more weight to ratings of items most similar to the target item.

Cosine Similarity between items i and j:
$$sim(i,j) = \frac{\vec{r}_i \cdot \vec{r}_j}{||\vec{r}_i|| \cdot ||\vec{r}_j||} = \frac{\sum_{u} r_{ui} \cdot r_{uj}}{\sqrt{\sum_{u} r_{ui}^2} \sqrt{\sum_{u} r_{uj}^2}}$$
Predicted rating for user u on item i:
$$\hat{r}_{ui} = \frac{\sum_{j \in N_i} sim(i,j) \cdot r_{uj}}{\sum_{j \in N_i} |sim(i,j)|}$$

2.5 User-Item Rating Matrix Example

Consider the following ratings matrix (1-5 scale, ? = missing):

User \ Movie M1 M2 M3 M4
U153?1
U24?52
U31135
U42345

3. Interactive Examples

Collaborative Filtering Similarity Calculator

Compute cosine similarity between two items based on user ratings:

User Item A Rating Item B Rating
U1
U2
U3
U4

Rating Prediction Calculator

Predict U1's rating for M3 using user-based CF:

Similar UserSimilarity to U1Rating for M3
U20.9655
U30.4233
U40.6584

User-Based vs. Item-Based Comparison

Aspect User-Based CF Item-Based CF
Similarity computed between Users Items
Best when # users < # items # items < # users
Stability Less stable (user preferences change) More stable (item similarity is static)
Sparsity handling Poor with sparse data Better with sparse data
Interpretability "Users like you enjoyed..." "Because you watched X..."

4. Numerical Solutions

Problem 1: User-Based CF Rating Prediction

Given: Predict U1's rating for M3.

User \ MovieM1M2M3M4
U153?1
U24?52
U31135
U42345

User Similarities to U1: U2=0.965, U3=0.423, U4=0.658

Step-by-Step Solution

Step 1: Identify relevant users

Users U2, U3, and U4 have all rated M3. Their similarities to U1 are 0.965, 0.423, and 0.658 respectively.

Step 2: Apply prediction formula

$$\hat{r}_{U1,M3} = \frac{sim(U1,U2) \cdot r_{U2,M3} + sim(U1,U3) \cdot r_{U3,M3} + sim(U1,U4) \cdot r_{U4,M3}}{sim(U1,U2) + sim(U1,U3) + sim(U1,U4)}$$

Step 3: Substitute values

$$\hat{r}_{U1,M3} = \frac{0.965 \times 5 + 0.423 \times 3 + 0.658 \times 4}{0.965 + 0.423 + 0.658}$$ $$= \frac{4.825 + 1.269 + 2.632}{2.046} = \frac{8.726}{2.046} \approx 4.26$$

Step 4: Interpret

The predicted rating is approximately 4.3 (or 4 when rounded to integer). Since U2 is most similar to U1 and gave M3 a high rating (5), the prediction is pulled toward the higher end.

Problem 2: Item-Based CF with Cosine Similarity

Given: Transposed matrix for item-based CF.

MovieU1U2U3U4
M15412
M23None13
M3None534
M41255

Task: Compute cosine similarity between M1 and M2 using shared users U1, U3, U4.

Step-by-Step Solution

Step 1: Extract rating vectors

For shared users U1, U3, U4:

  • M1 vector: [5, 1, 2]
  • M2 vector: [3, 1, 3]

Step 2: Compute dot product

$$\vec{r}_{M1} \cdot \vec{r}_{M2} = 5(3) + 1(1) + 2(3) = 15 + 1 + 6 = 22$$

Step 3: Compute magnitudes

$$||\vec{r}_{M1}|| = \sqrt{5^2 + 1^2 + 2^2} = \sqrt{25 + 1 + 4} = \sqrt{30} \approx 5.477$$ $$||\vec{r}_{M2}|| = \sqrt{3^2 + 1^2 + 3^2} = \sqrt{9 + 1 + 9} = \sqrt{19} \approx 4.359$$

Step 4: Compute cosine similarity

$$sim(M1,M2) = \frac{22}{5.477 \times 4.359} = \frac{22}{23.87} \approx 0.9215$$

This high similarity (0.9215) indicates M1 and M2 are very similar items.

Problem 3: Item-Based Rating Prediction

Given: Item similarities: M1-M2=0.9215, M1-M3=0.9567, M1-M4=0.5567. U1 rated M2=3, M4=1. Predict U1's rating for M1.

Step-by-Step Solution

Step 1: Identify rated similar items

U1 has rated M2 (sim=0.9215) and M4 (sim=0.5567). M3 was not rated by U1.

Step 2: Apply item-based prediction

$$\hat{r}_{U1,M1} = \frac{sim(M1,M2) \cdot r_{U1,M2} + sim(M1,M4) \cdot r_{U1,M4}}{sim(M1,M2) + sim(M1,M4)}$$ $$= \frac{0.9215 \times 3 + 0.5567 \times 1}{0.9215 + 0.5567}$$ $$= \frac{2.7645 + 0.5567}{1.4782} = \frac{3.3212}{1.4782} \approx 2.25$$

Note: This prediction (2.25) differs from U1's actual rating of 5 for M1, illustrating that item-based CF can produce inaccurate predictions when the user has rated few similar items.

5. Try It Yourself

Problem 1: Pearson Correlation

Compute the Pearson correlation between U1 and U3 using common movies M1, M2, M4:

MovieU1U3
M151
M231
M415

U1 mean = 3, U3 mean = 7/3 ≈ 2.33

Step 1: Compute centered ratings:

MovieU1 - 3U3 - 2.33
M12-1.33
M20-1.33
M4-22.67

Step 2: Numerator = 2(-1.33) + 0(-1.33) + (-2)(2.67) = -2.66 + 0 - 5.34 = -8.0

Step 3: Denominator:

√(4 + 0 + 4) × √(1.77 + 1.77 + 7.13) = √8 × √10.67 = 2.83 × 3.27 = 9.25

Step 4: Correlation = -8.0 / 9.25 ≈ -0.865

Interpretation: Strong negative correlation — U1 and U3 have opposite preferences!

Problem 2: Cold Start Problem

A new user U5 has rated only one movie: M2 = 4. Explain why both user-based and item-based CF struggle with this scenario, and propose a solution.

User-based CF struggles because:

  • We cannot compute reliable similarity with only one common rating.
  • Any similarity computed would be based on a single data point and highly unreliable.

Item-based CF struggles because:

  • We only have one rated item (M2) to base recommendations on.
  • The prediction for any new item depends entirely on M2's similarity to that item.

Solutions:

  1. Content-based filtering: Use movie genres, actors, directors to recommend similar movies to M2.
  2. Hybrid approach: Combine collaborative filtering with demographic information (age, location).
  3. Popular items: Recommend globally popular items until more ratings are collected.
  4. Explicit onboarding: Ask new users to rate a set of diverse items during signup.

6. Interactive Quiz

Answer all 5 questions. Click an option for instant feedback.

Your score: 0 / 5

7. Key Takeaways

8. Common Pitfalls